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Abstract 


A method for estimating velocity from two successive frames of motion smeared images 
is described. The problem is posed as a system identification problem. The underlying 
phenomenon of motion smearing is modeled as a hnear system with an appropriate 
transfer function. An algorithm for estimating the transfer function is derived. The 
motion vector is estimated from the support of the corresponding point spread function. 
Further, a new method for estimating depth and motion simultaneously is presented. 
The method uses the defocus and the motion-smear information present in the image 
frames. Both the methods are tested on simulated images. 
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Chapter 1 
Introduction 


It is well known that the visual system sums signals over time, (about 120 ms) in day- 
light. Although this summation has the advantage of enhancing visual sensitivity, it 
creates the potential problem of motion blur when viewing moving targets[l]. If the 
object is moving at a very high speed then, due to integration, motion blur is inevitable. 
It has also been reported that when the Human Visual System (HVS) is presented with 
an image, blurred due to motion, the amount of smear perceived by human observers in- 
creases with observing durations (up to 20-30ms), but at longer durations, the perceived 
smear actually decreases, i.e, the perceived image becomes sharper[l]. It appears that 
the HVS is performing a deblurring operation on the image. For deblurring, a knowl- 
edge of motion parameters is required. Hence, it can be concluded that motion smear 
information is probably used for motion sensation and motion deblurring in the HVS. 

In the context of image analysis, however, motion smear has always been considered 
as a degradation. A large amount of works have been done to remove this degradation 
[8]. But its positive aspect has always been neglected. Very little work has been done 
to exploit the positive side of this phenomenon, i.e, to use motion smear as a visual cue 
for image motion estimation.The first attempt of its kind has been reported in [2] where 
motion is estimated using motion blur. 

We establish a computational model that functionally emulates the behavior of the 
HVS, i.e, estimates image motion from motion smear information or, in short, “motion- 
firom-smear.” We further tried to investigate the possibility of estimating motion and 
depth simultaneously. For this we studied the defocus blur and motion blur simultane- 
ously. The work described in this thesis is an attempt to explore the positive side of two 
degradations (a) motion blur and (b) defocus blur, which one always tries to remove. In 
this work we extract two very important parameters from these degradation, i.e, motion 
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and depth. 


1.1 Motion Smear 

It is assumed that image is acquired by keeping the shutter open for an infinitesimally 
short duration of time, i.e, the shutter speed is very fast. This assumption holds good 
in many cases where image plane velocities are small relative to the image integration 
duration, but it fails in situations when available illumination levels are low. In such 
situations the shutter must be kept open long enough to ensure the integration of suf- 
ficient number of photons per pixel to produce images of adequate SNR. If the object 
moves with very high velocity, then it may cover appreciable distance during exposure 
time resulting in blurred image of the object. This motion blur is generally referred as 
motion smear in literature. 

1.1.1 Image Motion Estimation 

Previous research in motion estimation is based on two approaches : Gradient based 
and Region matching. Different variants exist for each approach, e.g, spatio-temporal 
approach for gradient-based techniques and recursive region matching for feature-based 
techniques. 

Most gradient-based techniques depend on the relationship between the spatial deriva- 
tives and temporal derivatives in time- varying imagery. In order to apply gradient-based 
techniques to practical situations the image motions must be small and apparent texture 
must be fine grained. But smaller motions are difficult to detect reliably in the presence 
of image noise. 

In region matching methods, a small sub-region in an image frame is considered. 
Within the sub-region, a search for the displacement which produces the best match 
among possible regions in the subsequent firame is evaluated. Feature-based techniques 
have the advantage of depending on complex image stmctures that can be detected in 
conjunction with noise suppression operations. Further, they can be applied to long- 
range motion situations, i.e., when the image plane velocities are large. 

1.1.2 Motion Estimation from Motion Smear 

Estimating motion from motion smear is fundamentally different from the families 
of existing motion estimation methods. It uses an alternative visual cue - motion 
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smear, while all other methods rely, directly or indirectly, on the conventional cue- 
displacement. This fundamental difference causes “motion-from-smear” to behave dif- 
ferently from the families of existing techniques. “Motion-from-smear” provides tech- 
niques applicable to motion of longer range than gradient-based techniques and, at the 
same time, applicable to imagery obtained using practical shutter speeds. 

In the thesis we first introduce a data acquisition scheme and a model for motion 
smear. Based on this, we develop a method to get motion parameters by utilizing the 
parametric system identification framework. We formulate the problem as that of find- 
ing an appropriate blurring operator, which transforms the local region image with less 
smear into the local region image with more smear information. The problem now can 
be expressed in terms of identification of a pertinent linear system. We further try to 
incorporate defocus blur in the model to estimate motion and depth simultaneously. For 
this a new scheme for data acquisition is introduced and by using parametric system 
identification approach we tried to get both the motion and depth parameters simulta- 
neously. 


1.2 Organisation of the Thesis 

The work carried out in this thesis is organized as follows. The work has been divided 
into two parts. The first part is concerned with motion estimation from motion-smeared 
images, while in the second part of the thesis, an attempt has been made to estimate 
motion parameters and depth simultaneously from motion smeared images which are 
also defocused by different amounts. 

The second chapter deals with the mathematical formulation of the problem of mo- 
tion estimation from two successive smeared images. The problem of motion estimation 
is posed as system identification and appropriate parametric transfer functions are de- 
rived. Finally analytical expressions are derived for coniputing displacements from the 
polynomial transfer function coefficients. 

Chapter three is concerned about system identification algorithms. We discuss the 
motion-smear model under noisy condition. We also discuss the effect of filter order 
and window size on the performance of the algorithm developed. 

In chapter four, we consider the restoration of blurred image from motion infor- 
mation obtained in the previous chapter. We study the problem of inverse filtering as 
applied to image restoration. The use of regularized filter for restoration is also dis- 
cussed. 
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Chapter five is devoted to simultaneous recovery of motion and depth. Firstly, we 
introduce a new data acquisition strategy to tackle the problem followed by solving the 
problem under the framework of system identification described in chapter three for 
motion estimation. 

In chapter six, we discuss the simulation and experimental results obtained by ap- 
plying the algorithms developed in the previous chapters to simulated test images. 
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Chapter 2 

Mathematical Formulation of the 
Problem of Motion Estimation from 
Motion-Smear 


2.1 Image Sequence Acquisition 

We adopt the same method for acquiring image sequences as explained in [2], It is as 
follows: 

Let f{x, y; t) denote an ideal, unbluixed image at time t. Further, suppose we have 
an imaging system with a non instantaneous shutter mechanism, i.e., a nonzero exposure 
time At, and denote the observed image by g{x, y; t, At). Thus, g{x, y; is 

obtained by the shutter opening at time and the sensor integrating the signal until 
time ti+i = ti + Ati^i+x for each {x, y). The subscripts of At denote the times when the 
shutter opens and the time when the shutter closes. We assume that the next observation 
begins immediately, so that at time = ^.+1 + ff(a:,y;ft+i,Af, + 1 , 1 + 2 ) is 

available for analysis. Fig. 2.1 illustrates the situation. 

2.2 Linear System Model for Motion Smear 

Motion smear is modeled as 

g{x, y; t,, At^+i) = / f{x, y; t)l{t; Ati,,+i)dt (2.1) 
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Figure 2.1: Graphical description of data acquisition. The original unblurred image at 
time ti is denoted hy The shutter remains open for time interval The 

blurred image is denoted by g{x, y\ U, s(x, y; U, At, , 1 + 2 ) would be the image 

acquired if ther camera shutter were kept open from ti to t,+ 2 - 


where Atj^i+i) is the transmission coefficient of the lens/filter system. If the 
image motion is translational then 

/ oo roo 

/ f{a,^]ti)6{x - v^it Vy{t - t,) - I3)dadp (2.2) 

>00 J —00 

where t G (t„ t,+i] and v = {vx, Vy) is the image motion vector at {x,y). From equa- 
tions (2.1) and (2.2) we get 
g{x,y,U,At^+x) = 

AUt+i /-oo / (®5 P'i — Vx{t — t,) — cx,y — Vy{t — ti) — P)d(xdP) 

l(t, ti, Ati^i^i)dt 


f- 00 1-oo /(a= /5; it)( /**'■"' 5{x-Vxit-ti)-a,y-Vy{t-t^)-^)l{t; t„ At,,i+i)dt) 


dadp 


If 

h{x,y,a,p-,ti,Ati,i+x’,\) = 

— / 5(a: - Vx{t - ti) -a,y- Vy{t - t^) - 0)% ti, At,,i+i)dt (2.3) 

then 

/ OO roo 

/ f{a,^-,U)h{x,y,a,p-,t„Ati,i+i,-v)dadp (2.4) 
-00 •/— 00 

This is a generalization of the linear system model for motion smear that is gener- 
ally used in image restoration. Here h{.) is a blurring operator known as point spread 
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function (PSF). We take the sensor gain l{.) as unity over the shutter opening duration, 
which is the well known boxcar model used in image restoration [8]. Then the PSF 
equation (2.3) reduces to 

r*t+i 

; v) = / 5{x - v^{t - t,) - a,y - Vy{t - 1^) - P)dt (2.5) 

'ftx 

The motion vector is known if h{.) is available because the support of the h{.) is 
uniquely determined by v. In general, h{.) is space-varying in nature but it may be 
assumed as locally space-invariant. It is also assumed in the above model that v doesn’t 
change during (ti,ti+i]( uniform motion). 

Consider two successive motion smeared frames at z = 0, 1. We further assume that 
V is constant over time intervals of the order of Atoi + Ati 2 . 

If we take the space-invariant linear system model, we get 

^(a:, y; to, Atoi) = fix,y,to)-kh{x,y,to,AtQi,v) 

= f{x,y;to)'khoi{x,y,^/) (2.6) 

and 

g(x,y,ti,Ati2) = fix:y,ti)i^h{x,y,tuAti2]v) 

= /(a;,y;ti)*/ti2(x,y;v) (2.7) 

In the space-invariant case, equation (2.2) simplifies to 

f{x,y,ti) = f{x,y,to)-*^5{x-VxAtoi,y -VyAtoi) 

= f{x,y;to)'k6{x- Lx,y - Ly) 

where 

— v^Atoi 

Ly ~ VyAtQI 

Now we create the sum signal defined as 

s{x,y;to,Ato 2 ) = gix,y, to, Atoi) + g{x,y,ti, Atn) (2.8) 

or briefly, 
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Original Image 


Motion 

smeared 

images 


Sum of two 
smeared images 



SQ2 = 9oi + pi2 

= hoi + fi-khi2 

= fo-^hoi + *^ 12 ; where /i' = 6{x - L^,y - Ly)] 

— fo'^ i^oi + h' -k hi2) 

S 02 = fo-khQ2 (2.9) 

where, ho 2 = h-oi +h' -k hi 2 

Here we have used the shorthand notation ^ 01 . 9 x 2 S 02 , /toi. hi 2 for 5(x,y;to,^^oi). 
g{x,y\ti, M 12 ), s{x,y,to,AtQ 2 ), hQi{x,y;v), hi 2 (x,y;v) respectively, wc note that 
S 02 corresponds to the blurred image acquired if the camera shutter were kept open 
from to to t 2 (Note: Here we left averaging factor | in equation (2.9) for simplicity). 
Since the PSFs are space varying, thus all the above results are only locally valid. The 
Fig. 2.2 illustrates the above equation. 

2.3 Motion Estimation as System Identification 

In frequency domain we can write from equations (2.6), (2.7) and (2.9) 

Croi{0,x,0'y',tQ, Atoi) — Fo(Q3;j^y)Hoi{^x}^yt''^) (2.10) 
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goi(x>y) 



So2(x,y) 


Figure 2.3: The block diagram of the underlying system 

<S'o2(f^2) ^0) '^^ 02 ) ~ Fo(f^a:j ^j/)-^02(^a:) v) 

= Fo(Q,X)^y){Hoi{^Xi^y) + H' {D,xO,y)Hi2{^Xi^y)) 

( 2 . 11 ) 

Let the two exposure duration Afoiand Ati 2 be the same, we have 


Hoi{nx,ny) = H,2i^x,^y) = 


where, Q.L = D-xLx + ^yLy 


(2.12) 


Further, 


H'{Q.x, ^y) - FT{h'(a:, y; v)} = exp{-jQ,L) 


Hence from equations (2.11), (2.12) and (2.13) we have 

SQ2{'^x,^y) = Fo(fix,^^j,)-ffoi(^^x,^y)(l + exp(-^m)) 

= (?oi(fix,^^y)(l + exp(-jf2L)) 


(2.13) 


(2.14) 


If 


HiQx^^y) = l+expi-jQL) 


then from equation (2.14) we have 

= H{Qx,^y)Goi{'^x,^y) 


H{Clx, fiy) 


jSo2(t^a, ^y) 
(^01 (^X) ^y) 


where 


or, so 2 (a:,y) = h{x,y]v)-k goi{x,y) 
h{x,y,v) = 5{x,y) + 6{x - Lx,y - Ly) 


(2.15) 


(2.16) 


(2.17) 

(2.18) 
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Figuxe 2.4: System identification 

Fig. 2.3 shows the block diagram where goi{x, y) is the input of a linear system 
whose output is SQ 2 {^,y) (Note: the time dependence of the notations for poi(-) ^nd 
so 2 (-) have been dropped for convenience). 

Equation (2.17) represents a linear system with h{x, y\ v) as the impulse response. 
Ass umin g local space invariance, the motion estimation problem can be framed as a sys- 
tem identification problem, where given the observations of the input {poi} and output 
{S 02 }, we have to identify certain parameters of the underlying system. The derivation 
of the relevant system description firom the observed images needs a model to describe 
the system and an identification procedure to estimate the parameters of the model. This 
in turn will provide the relevant information we wish to infer about the system. 

2.4 Parameterization of the Transfer Function 

In order to estimate motion, we need to approximate H{Q,x,^y) by some parametric 
function i.e find a function B{Q,x, Cty) which approximates ^y) by minimizing 

an appropriate cost function through an identification algorithm as shown in Fig. 2.4. 

Equation (2.16) models the underlying process in the continuous domain. However, 
in practice, only digitized images are available, therefore we have to find an appropriate 
parameterization of the transfer function which will enable us to obtain the parameters 
from those digitized images. Let goi{k, 1), so 2 {k, 1) and /i(A:, 1) be the sampled signals 
corresponding to goi{x,y) ,so 2 (,x,y) and h{x,y,v) respectively. The sampled signals 
are related in Fourier domain as 
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(2.19) 






(jOl(wi, W2) 

where 5o2(wi,W2), Goi(wi,W 2 ) and H (uj lyOj^) are the Fourier transforms of the se- 
quences so 2 (^) 0’ 5 oi(^) 0 ^(^5 0 respectively. Now we reformulate the problem as 

that of approximating the transfer function H{oji,(j 02 ) of the underlying system with an 
appropriate parametric transfer function B{uji,uj 2 ). 

Since equation (2.15) suggests that the unknown transfer function H{uji,u} 2 ) is of 
polynomial form, it is approximated with non-causal polynomial transfer function with 
bmn as coefficients, i.e. 


M M 

B(a;x,a;2)= ^ E (2.20) 

2.5 Analytical Expression for the Displacements 

We have, 

H{(jJi,u} 2 ) = FT{h{k,l)} — FT{h{mAT,nAT)y = 1 -f exp(— ywL) (2.21) 

where, ujL = UiL^ H- 0 J 2 Ly 

Now taking the partial derivative of equation (2.21) and evaluating it at cui = a ;2 = 0 , 
we get 


dH{uJi,U2) 


dooi 


ia;i=Lt72=0 


= {-jL^)iexp{-jujiL^) exp{-jaj2Ly))\^^^^^Q 
= {-jLx) 


dH{aji,u}2) 


du)2 


ct;i=ti;2=0 


(-jL,)(exp(-jwiL,)exp(-jU2Lj))|„__^^ 

(-jX,) (2.22) 


If we take the partial derivative of B(aji, ^ 2 ), we get 


9J5(a;x,a;2) 


doJi 


lc*;i=a;2=0 


M M 

= -J( E E ^bran) 

m^—M n=-—M 


-A . 
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dB{ij0i,U2) 


duj2 


ijJl=(jj2=0 


M M 

-Ji Y, £ 

m——M m^—M 


(2.23) 


Hence from equation (2.22) and (2.23) we get analytical expression of displacements 
during shutter opening time as 


M u 

m—~M n=—M 
M M 

Ly X] (2.24) 

n=—M 


One way of interpreting equation (2.24) is that displacements are the first moment of 
transfer function(i.e h{x, y; v)) about x and y axis respectively in our case. Hence once 
the coefficients of the transfer functions are approximated by some suitable algorithm, 
we can easily estimate displacements analytically. 
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Chapter 3 

Identification of Transfer Fnnction 
Coefficient 


System Identification or the process of constructing an approximation to the underlying 
transfer function of a system from the input-output measurements has a rich history and 
is a well studied problem. It is one of the central tools used in signal processing and 
control theory. Construction of an approximation to the underlying transfer function of 
a system requires four things, viz, 

• The Input-Output measurement record. 

• An appropriate Model for the transfer function. 

• A cost function which characterizes how close the model is to the transfer func- 
tion. 

• An algorithm to identify the parameters by minimizing the cost function. 

The extraction of any relevant description about the system is limited by the input- 
output measurements. These measurements may be contaminated by noise or there 
may not be enough information present in them for proper identification. These factors 
affect the accuracy of the identified parameters. 

As discussed in the previous chapter, the appropriate model for the transfer function 
to be approximated is a non-causal polynomial in our case. Though it can also be 
approximated by rational transfer function, but in our work we used only polynomial 
transfer function approximation. 
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The cost function quantifies the closeness between the model and the underlying 
transfer function. Assuming that a model is available, a best approximation is obtained 
by minimizing the cost function in the parameters. In general, the cost function is 
chosen such that it is convex and has a unique global minimum. One such function is 
the quadratic error criterion in which the sum of the squared errors is minimized, which 
is known as the least mean squares error criterion. 

The parameters are obtained by minimizing the cost function through an appropri- 
ate algorithm. The choice of algorithm is determined by its ability to converge to the 
global minimum and by its computational complexity. In the following we develop 
equation error algorithms for the minimization of the quadratic cost function to identify 
the coefficients of the polynomial transfer functions. 

3.1 Equation Error Method 

Consider the parametric transfer function defined by the equation (2.20). Substituting 
it in (2.16) and rearranging the terms we get 

_ MM 

So2(cui,u;2) = 52 ^mnGoi.(wi,a;2)exp(-jwim)exp(-jw2n) (3.1) 

m=—M n=—M 

where 5o2(u;i, 0 ^ 2 ) is the reconstruction of 502 (^ 1 , U 2 ). Taking the inverse Fourier 
Transform, we have 


M M 

— 52 52 ^mnQoii.^ TTljl Tl) (3.2) 

m=~M n=—M 

for all k, 1 

where the output is modeled as moving-averages of the input. We now define equation 
error signal [3] as 

e{k, 1) = SQ 2 {k, 1) - SQ 2 {k, 1) 

M M 

= 502(^j 0 52 5 y ^mn90l(J^ TTl^l Tl)] (3.3) 

for all k, 1 

The flow of the equation error signal is shown in Fig.3.1. The equation error can be 
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mv^2> 







B(2 

• 1 ,^ 2 ) 

1 



|e(k,l) 

Figure 3.1: The flow diagram of equation error signal 



written in terms of lexicographically ordered vectors as 


or if a* 


[00 1 


e{k,l)=[-h'^ l] 


S02ik,l) 


0 0] then equation (3.4) can be rewritten as 


(3.4) 


e{k, 0 = [ — ] 


goi (kj) 
S02 {k,l) 


(3.5) 


where b is a vector of length K, obtained by stacking the columns of the coefficient 
matrix into a vector form given by 


b^ = [b-M,-M bM,-M -—bofi b_M,M 

and the vectors goi, S02 is obtained by stacking the columns of Lx L matrix extracted 
from the input local region image around the point {k, 1) with L = 2M + 1 and K = L^. 

goi^(fc)0 ~ Isoiik M,l + M) goi{k — M,l + M) .... 

goiik, 1) ...goi{k + M,l-M) ... 

....go,{k-M,l-M)] 


So2^{k, t) = [5o2(fc "b M, I + ilf) SQ2{k — M, I + Af) — 


15 





So2{k, 1) ...S(y2{k + M, I — M') ... 

....S02{k- M,l-M)] 


The variance of the equation error given by equation (3.5) is easily found as 


where 


-b^ a^l 

■^5s(^3 0 R'p5(^j^) 

' -b ’ 

J 

Kl{k,l) Iiss{k,l) _ 

a 


Iigg{k,l) = E[goi{k,l)goi'^{k,l)] 

Ilg,{kJ) = E[goi{Kl)So2'^{k,l)] 

K,,(k,l) = E[so2{k,l)so2^{k,l)] 


(3.6) 


(3.7) 


where E{.) is the expectation operator and Rgg , and Rs, are the signal corre- 
lation matrices. Further, assuming the signals of interest to he stationary, the spatial 
dependence of correlation matrices will be suppressed. 

We assume that the input image over a local region around a given point is stationary, 
ergodic and persistently exciting of sufficient degree [3], such that Rgp is invertible. We 
can then use a matrix identity 



Jigs 


I 

0 ' 


^99 

0 

1 

R55 


, -^gs^gg 

I 


0 

r„-r 5 ;r-%. 


I Rjg'Rps 


(3.8) 


which gives a block triangular LDU factorization. All the matrices in the above equation 
are of dimension K x K . At first sight, this factorization looks formidable, but of 
particular interest is a term in the center factor: 


Rfl/g — (3.9) 
This is called the Schur complement of R with respect to Rj,g . Note that R is 


R = 


Rj, B... 
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From equations (3.8) and (3.9) we have. 


E[e^{k, 1)] = a^R,/ga + [-b + + R-'R^^a] (3.10) 

Now minimizing equation (3.10) with respect to the vector b gives 

b = ^gg Rgs^- 

or, b = R“j^rps where = Rp^a (3.11) 

Once we obtain vector b as discussed above we can get velocity vector v using 
equation (2.24). 


3.2 Dealing with Noise 

The model described in the previous chapter is the noise free case i.e, we have assumed 
the observed images are noise free. Since such an assumption is impractical in real situ- 
ation, we consider the model for motion-smear under noisy image frames and show how 
well suited the equation error identification algorithm is to approximate the polynomial 
transfer function under noisy condition. 

Let us consider the noise corrupted versions of g{x, y\ Atj^i+i) as, 


gn{x, y; to, Atoi) = 9 {x, y; to, Atoi) -F n{x, y; to) (3.12) 


and 

y„(a:,y;ti, Ati 2 ) = g{x,y;ti,Ati 2 ) +n{x,y;ti) (3.13) 

where n{x, y\ to, Atoi) and n{x, y; ti. At 12) are zero-mean, white Gaussian noise. 

The noisy sum signal Sn{x, y\ to, Ato 2 ) is defined as 


so2„(a:,y) = Soi„(a;,y) + 5i2„(a;,y) 

= Poi(a:, y) -f- gv 2 {x,y) + no(a:, y) -F ni{x, y) 

= SQ2{x,y) + n2{x,y) (3.14) 

Here we have used notations so 2 „(-) , 5oi„(-) , Pi 2 „(-) ,tT'o(-) and ni{.) to denote for 
noisy version of the signals and noise. We leave the time dependence of notation for 
convenience. Note that n 2 {x, y) is a zero-mean Gaussian noise. 
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From equation (2.17), we see that under noise free condition so 2 (a;, y) is given as 


502 ( 2 :, y) = h{x,y,v)-kgQi{x,y) (3.15) 

Hence, we can model the system in the noisy case as 

SQ2n{x,y) = h{x,y,v)* goi{x,y) + n2{Xjy) (3.16) 

The acquired noisy gQi„{x,y) needs to be denoised by some prefiltering operation to 
get goi (x, y) = yoi ( 2 :, y). Two feasible solution are considered fo'r the noisy case: 

1 . Dcnoise both the input yoi {x, y) and output S 02 {x, y) frames and then do the mo- 
tion cstimation.This is the noise free case as discussed in the previous chapter. 

2. Denoise the input frame yoi„ ( 2 :, y) to get ^ 01 ( 2 :, y) — 501 ( 2 :, y) and then approxi- 
mate the motion parameters under system identification frame work, using model 
described by equation (3.16). 

The system model described by equation (3.16) is the one which is generally used in 
system identification by equation error method[3]. The method discussed in the pre- 
vious section is one of its case when n 2 (x,y) = 0. In the following sub-section, we 
will show that if we adopt the model described by equation (3.16), then the estimated 
polynomial coefficient b will remain unaffected by the output noise n 2 (x, y) . 


3.2.1 Estimation of polynomial transfer function coefficients 

Fig. 3.2 shows the signal flow graph of the equation error identifier. Here H{zi, Z 2 ) 
is the unknown polynomial transfer function to be identified, driven by a preprocessed 
input {goi{k, 0} and whose output {so 2 (^, 0} is corrupted by a Gaussian disturbance 
{n^ik, ()} which is assumed statistically independent of the input. 

The equation error whose variance is to be minimized is 


e{k, 1) = — ] 


goi 

S02e 


(3.17) 


where b^, goi and So2„ are lexicographically ordered vectors as discussed earlier. Note 

that here, = [00 1 00]. The variance of the equation error is easily 

found as 
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Figure 3.2: Signal flow graph of equation error identifier 


E[e\k,l)]=[-h^ a^] 


^93 

RJ 

gsn 


^SnSn 


(3.18) 


with 


[ioiloi] = ^ [gOlgOl^] = Rflp , 

^ [gOlS02n^] = ^ [g0l(S02 + 112)^] = Rgs, 


^Sn«n — ^ [s02o®02n'^] — R-Sfl "F ^nj^J 

Note that the spatial dependence of the correlation matrices have been suppressed as 
earlier we did with stationary assumption of the signals. 

Now equation (3.18) is further simplified as 


E[e^{k,l)] = [ — ] 




■gs 


rT. r . 


'‘gs 


-b 

a 


+ ‘^n2a^a 


(3-19) 


Since a^a = 1, hence equation (3.19) may be rewritten as 


E[e\k,l)]=[-h'^ a^' 


B.I Uss 



(3.20) 


As we see from equation (3.20) that the presence of noise now adds only a constant 
term to the equation error variance, so the polynomial coefficient vector b^, obtained 
by minimizing the error variance, does not vary with the noise power This is the 
advantage of using equation error identifier for our estimation purpose. 

As derived earlier, the polynomial coefficient is obtained by minimizing the equation 
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(3.18) which gives 


*> = = K-mRs. (3-21) 

Hence the estimation of coefficient is not affected by background noise power when 
we adopt the model defined by equation (3.16). 

3.3 Filter Order and Window Size 

The two pai'ameters of particular importance in this procedure are 

1 . The choice of filter order determined by M and 

2. The window size iV x of the local region. 

The choice of M to determine filter order plays a crucial role in finding bmn- Tlie 
automatic model order determination in 2-D case through eigen analysis of Schur com- 
plement [4] is difficult owing to non-causal nature of model. However, if we assume 
that maximum displacement of the object is known apriori, then we may be able to fix 
the model order. A correct choice of M is the least value at which MSE reaches its 
minimum. This is shown in the chapter 6. The assumption of ergodicity requires a large 
number of samples (pixels). The resulting increase in window size conflicts with the 
assumption of space invariance over a local region. An optimal choice of window size 
is therefore cmcial. 

If we consider 1-D case of system identification and let H{z) to be approximated 
by B{z) and both are rational functions of degree Mi, then in order to identify the 
unknown transfer function H{z) i.e, B{z) = H{z) the input signal {u(n)} should be 
persistently exciting of degree 2Mi + 1 (=the number of free parameters) or greater[3]. 
If {u(n)} is persistently exciting of degree less than 2Mi + 1, then exact identification 
of the system is not possible . This concept can be extended to 2-D case and we have 
taken N = AMmax + 1, where Mmax is the value of M corresponding to maximum 
displacement, so that in the case of maximum displacement, transfer function can be 
identified. 


3.4 The Complete Algorithm 

The overall algorithm for estimating motion from motion-smear model is: 
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1. Select a location {xt,y^) in the images and define an N x N local region image 
around that location. 

2. Obtain the correct value of M as explained in the section (3.3) so that the approx- 
imating transfer function is of sufficient order. 

3. Estimate polynomial transfer function coefficient vector b, using equation error 
identification algorithm. 

4. Using these coefficients we calculate displacement analytically as discussed in 
chapter 2. 

5. Move to the next image location (xi+i, yt+i) and repeat the above to get the mo- 
tion parameters at that point. 
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Chapter 4 


Image Restoration 

4.1 Image Restoration as Inverse filtering 

We have seen in the previous chapter that motion smeared image can be modeled as 

g{k,l) = fo{k,l)-kb{k,l) (4.1) 

where the degraded image g{k, 1) is the result of convolving the original image fo{k, 1) 
with the blurring operator b{k, 1). The problem of restoring original image can be classi- 
fied into two categories. One is the deconvolution problem, in which b{k,l) is assumed 
to known. The second is the blind deconvolution problem, where b{k, 1) is not known 
and must be estimated from available information. Once the blurring operator is esti- 
mated or it is known, we can easily reduce the blur by inverse filtering . From (4.1) we 
obtain. 


G(a;i,a?2) = Fo(wi,a;2)B(wi, W2) 


or 


Fo(a;i,a;2) 


G{uJi,UJ2) 
B{ui, 0J2) 


(4.2) 


Fig. 4.1 illustrates the equation (4.2). The inverse filter in Fig. 4.1 tends to be very 
sensitive to noise . When B(ciJi, ^ 2 ) is very small, large, and small noise 

in the frequency regions where ^ is very large may be greatly emphasised. 


00 



g(ni,n2) 

1 

^(n^n^) 

B( coj .(o^) 




Figure 4.1: Inverse filtering for image restoration. 

4.1.1 Regularised inverse filter 

An inverse problem is characterised as ill-posed when there is no guarantee for the 
existence, uniqueness, and stability of the solution based on direct inversion. Several 
methods have been studied to solve the inverse problem. One such method of lessening 
the noise sensitivity is to use the regularised inverse filter. We define a regularised 
inverse filter as 


H{uJi,C02) = 


B*{Ui,UJ2) 


(4.3) 


B{uJua}2)B*{uji,U2)+'y 

where 7 is called the regularizing parameter . When 7 0, it becomes the inverse 

filter as defined in (4.2). Thus, if we choose 7 to be very small, we can restore the 
original image without having any noise amplification problem. 


4.2 Image Restoration from Motion Smear Model 

Once the blurring operator is estimated, we can restore the original image as discussed 
in the previous section. As we see from (2.5), the PSF function is given as 

h{x, y; ti, v) = 5(a: - - ti),y - Vy{t - ti))dt (4.4) 

Its Fourier transform is calculated as 

2 - nr. , ^ .. — 

if Hj,; v) = — . exp-^^ .sm(— ) (4.5) 

where = ^xBx + ^yBy 

Thus blurring operator is known when the displacements are estimated correctly. 
So once we are able to estimate motion from our motion-smear model, we can restore 
original image as discussed above. Thus motion-smear model can be used as image 
restoration method. 




Chapter 5 

Simultaneous Estimation of Motion 
and Depth 


5.1 Brief Introduction 

Determination of object distance from defocused images has been investigated by many 
researchers. It has been found that the defocus operator contains the depth information, 
so it can be exploited to estimate the distance of an object. Using the same idea we have 
incorporated the defocus information in our motion-smear model to estimate depth and 
motion parameters simultaneously. In the following section a brief concept of depth 
from defocus (i.e, how defocus information helps us to obtain the distance of an object) 
has been introduced. Then we try to incorporate the defocus information in our motion- 
smear model. For this a new data acquisition strategy has been introduced which helps 
to get a system model which contains both the motion and depth information. 

5.1.1 Depth from Defocus 

Fig. 5.1 illustrates the image formation in a simple camera. Let P be a point on a visible 
surface in the scene and p be its focused image. P and p is related by the well known 
lens formula as 


1 

7 


1 1 

— I — 
u V 


(5.1) 


where u is the object distance and v is the image distance. If P is not in focus it gives 
rise to a blurred image. According to geometric optics, the blurred image of P has the 
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Figure 5.1: Image formation in a simple camera system 


same shape as the lens aperture but scaled by a factor. Let the blurred image of the 
point P be /i(x, y). This h{x, y) is known as the point spread function. If we consider 
camera systems having a circular aperture, then the blurred image of a point on the 
image detector is circular in shape and is called the blur circle. Let R be the radius of 
the blur circle and D be the diameter of the lens aperture, and s be the distance from 
the lens to the image detector plane. From the geometry it can be shown that 


R= s 


(5.2) 


£) [1 _ 1 _ 1 

J « 

According to geometric optics, the intensity within the blur circle is approximately 
constant. If we further assume the camera to be a lossless system we get 


h{x,y) = 


^ ifx^ + y'^<R^ 
0 otherwise 


(5.3) 


where h{x, y) is the PSF of the camera.Taking diffraction and non-idealities of lenses 
into account, an alternative model has been suggested for the intensity distribution given 
by two dimensional Gaussian function 


, N r 1 » ty- /i- >i\ 

/i(x,y;u) = ^e (5.4) 

where a is the spread parameter such that a = kR for A: > 0. is a constant of 



proportionality characteristic of the given camera and mostly taken as ^[6]. 

If the radius i? is a constant over some region on the image plane, the camera acts 
as linear shift invariant system. Therefore the observed defocused image gd{x, y) is 
the result of convolving the corresponding focused image f{x,y) with camera’s pomt 
spread function h{x, y\ a), i.e, 

ffd(x, y) = h{x, y\ o) -k f{x, y) 

The point spread functions defined above are only two specific examples, it can be 
of other form which is characterized by the spread parameter a which is the standard 
deviation of the distribution of any function h. For the circular point spread function 
spread is given by <7 = Hence once we know the spread a of the defocus operator 
h(x, y; a) we can easily find out the depth using equation (5.2) which shows that the 
spread linearly depends on the inverse distance 

5.2 A General Definition for the Spread Parameter 

The form of the spread function determines the definition of spread. This is generally 
assumed to be one of the point spread functions as defined in (5.3) or (5.4). However in 
practical optical systems, due to diffraction effects, aberrations etc., the effective point 
spread function can become quite complicated. Usage of circular or Gaussian function 
may result in inferior depth estimates because the definition of spread is intimately 
related to their form. If the point spread function of the optical system happens to be 
different because of other effects, then the estimate of spread is bound to get affected. 
Measuring the PSF from the camera itself, though, in general provides better results, 
but it is quite complicated and experiments have to be conducted with care. Therefore 
it would be desirable to have the definition of the spread independent of the form of the 
point spread function. 

Def. : Let h{x, y; a) be any general defocus operator. We then define the normalized 
spread parameter as its second moment given by 

0 ^ = ^ f°° f°° [(x - xf + (y - yf] h{x, y; a)dxdy 

Ah J-00 J-00 ^ 

where ($, y) is the location of the center of mass of h{x, y; <r) defined by 


(5.5) 



rao TOO 

“ TF / / xh{x,y\a)dxdy 

^ J-ooJ-oo 


roo TOO 

y = / / yh{x,y;a)dxdy 

j-ooJ-00 

(5.6) 

roc roo 

■4/1= / / h{x,y,a)dxdy 

J -00 J -00 

(5.7) 


In any praclical iiptical system, the above dcfocus operator has the following properties: 

• 'Fhe (Jetbeus operator is rotationally symmetric. 

• Assuming the camera to be lossless system 

/ DO roe 

/ h{x,y,a)dxdy = l ( 5 . 8 ) 

-00 i—OO 

Even though the camera’s aperture is not circular the spread of the defocus operator 
is linearly related to u“^[6]. Hence, once we are able to estimate spread of the defocus 
operator by any means then the distance of object is easily found out. 

5.3 A New Data Acquisition System 

We take two successive frames goi {x, y; Afoi) and 512 ^ (x, y; Ati 2 ) by an imag- 
ing system with a non instantaneous shutter mechanism. The first frame is focused but 
it undergoes motion-smear during the exposure duration Aioi of th® imaging system. 
While during acquisition of the second frame, defocus is introduced dehberately so that 
the second frame gets defocused as well as motion smeared. Note that both the frames 
are taken in continuation. We then create a sum signal So 2 <j {x, y, cr^', to, Aio 2 ) from these 
two frames. Fig. 5.2 illustrates the method of acquiring two successive frames. 

5.4 Mathematical Formulation of the Problem 

Let goi (x, y) and gn^ (x, y) be two successive frames acquired as mentioned above (note 
that the time and spread dependence notation has been dropped for convenience) . If 
/o(x, y) is the original image then for linear space invariant system we have 



X 


roo roo 

^ ~ / / xh{x,y\o)dxdy 

J-ooJ-oo 


rOO rOO 

^ / / yHx,y,c)dxdy 

J^ooJ-oo 

(5.6) 

TOO roo 

Ah= / h{x,y;a)dxdy 

^-“00 J—oo 

(5.7) 


In any practical optical system, the above defocus operator has the following properties: 


• I'hc defocus operator is rotationally symmetric. 

• Assuming the camera to be lossless system 

/ DO roc 

/ h{x,y,a)dxdy = l (5.8) 

“00 J—oo 

Even though the camera’s aperture is not circular the spread of the defocus operator 
is linearly related to Hence, once we are able to estimate spread of the defocus 

operator by any means then the distance of object is easily found out. 


5.3 A New Data Acquisition System 

Wc take two successive frames poi V', ^o, Atoi) and {x, y; oi; ti, Atu) by an imag- 
ing system witli a non instantaneous shutter mechanism. The first frame is focused but 
it undergoes motion-smear during the exposure duration Atoi of the imaging system. 
While during acquisition of the second frame, defocus is introduced deliberately so that 
the second frame gets defocused as well as motion smeared. Note that both the frames 
arc taken in continuation. We then create a sum signal so 2 d(a:, y; 02 ; to, Ato 2 ) from these 
two frames. Fig. 5.2 illustrates the method of acquiring two successive frames. 


5.4 Mathematical Formulation of the Problem 

Let yoi {x, y) and (x, y) be two successive frames acquired as mentioned above (note 
that the time and spread dependence notation has been dropped for convenience) . If 
fo(x, y) is the original image then for linear space invariant system we have 



poo poo 

^ = TT / / yi a)dxdy 


poo poo 

y = TT / / yh{x,y,a)dxdy 

(5.6) 

poo poo 

^h — \ / h{x,y,a)dxdy 

J — OO j —OO 

(5.7) 


In any practical optical system, the above defocus operator has the following properties; 

• The defocus operator is rotationally symmetric. 

• Assuming the camera to be lossless system 

/ oo poo 

/ h{x,y]a)dxdy =1 (5.8) 

-OO J—oo 

Even though the camera’s aperture is not circular the spread of the defocus operator 
is linearly related to u“^[61. Hence, once we are able to estimate spread of the defocus 
operator by any means then the distance of object is easily found out. 

5.3 A New Data Acquisition System 

We take two succc.ssive frames goi {x, y\ to, Aioi) and 1 ^ 12 ^ {x, y, Ati 2 ) by an imag- 
ing system witli a non instantaneous shutter mechanism. The first frame is focused but 
it undergoes motion-smear during the exposure duration Atoi of the imaging system. 
While during acquisition of the second frame, defocus is introduced deliberately so that 
the second frame gets defocused as well as motion smeared. Note that both the frames 
arc taken in continuation. We then create a sum signal So 2 d (aj, y, era; to, Ato 2 ) from these 
two frames. Fig. 5.2 illustrates the method of acquiring two successive frames. 


5.4 Mathematical Formulation of the Problem 

Let goi {x, y) and gn^ {x, y) be two successive fiames acquired as mentioned above (note 
that the time and spread dependence notation has been dropped for convenience) . If 
fo{x, y) is the original image then for linear space invariant system we have 
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here f j(x,y;ti)=fi(x,y;ti)* h^(x,y); 

Figure 5.2: Graphical description of data acquisition 


9oi{^.y) = fo{x,y)-^hQi{x,y;Y) (5.9) 

= fiA^,y)'>^hi2{x,y,Y) (5.10) 

where /ioi(a:,y; v), hnix.y;^) are the motion smearing operator as explained in the 
previous chapter. Further, in the space-invariant case /ij(x, y) can be written as 

fiA^,y) = fi{^,y)-^hd{x,y;a) and (5.11) 

fi (x, y) = /o{x, y) * 5(x -Lx,y- Ly)\ with Lx = UxAtoi, Lj, = UyAioi 

(5.12) 

From (5.12) y) can be rewritten as 

fiA^,y) = h{x,y)i<h'{x,y)-khd{x,y,ay, (5.13) 

with h'{x, y) = 6{x - Lx, y — Ly) 

If we create a sum signal by adding the two frames as 

So 2 <, {x, y) = 5oi(a:, y) + {x, y) 

then from (5.9), (5.10) and (5.13) we have 

SQ2i{^^y) = 9Qiip^y) + 9i2A^^y) 

= /o(a:, y) * hQi{x, y; v) + fo{x, y) ★ h!{x,y) ★ hd{x, y, a) -k h^ix, y; v); 
= fo{3;,y)-^{hi{x,y,^) + h'{x,y)-khd{x,y,a)ichi2{x,y,Y)) (5.14) 


r\n 



In discrete domain, (5.9) and (5.14) can be expressed as 
goi{k,l) = foi^k,!) -k hQi{k,l) 

So2^{k,l) = foik,l)kho2^{k,l) (5.15) 

whcrc/io2rf(A:,0 = hQi{k,l) + h'(k,l)k hd{k,l]a)khi2{k,l) (5.16) 

In frequency domain, we have from (5.16) 

C?oi(^i!i^ 2 ) = Fo{<^i,^ 2 )H{u}i,u} 2 ) and 

= ■^o(Wi,CU2)(i?oi(Wi,a;2) + H\(j0i,(V2)Hd{Ui,U2)Hi2{Ui,LO2)) 

(5.17) 

Since the two exposure durations Atoi and A.ti 2 are same, we have 

Hqi{ui,U} 2) = Hi2{0Ji,Ul2) (5.18) 

From (5.17) and (5.18) we have 

.So2rf(Wi,W2) = Fq{u;i,U)2)Hqi{uJi,U} 2){1-+ H'{u:i,UJ2)Hd{u}l,U!2)) 

= (7oi(^1iW2)(1 + H'{u!ijU:2)Hd{cOi,UJ2)) 

.So2^(aji, W 2 ) = Goi{uJi,u]2)H{u}i,u)2)‘, (5.19) 

where H (wi, CU 2 ) = 1 + u} 2 )Hd{uuU} 2 ) 

Here 012 ) is given as 

H'{ujuuj 2 ) = FT{h'{k, 0} = exp(-jwL); (5.20) 

where wL = ojiLx + oj 2 Ly 

Etjuation (5.19) can be written in spatial domain as 

So2d{k}l) ~ (5.21) 

where 

h{k, 1) = 6{k, 1) + h'{k, 1) k hd{k, 1) (5.22) 

Fig. 5.3 illustrates the equation (5.21). Now the problem reduces to system identifi- 
cation problem in which we have to estimate H{zi, Z 2 ) from the input-output data. As 
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{•‘igurc 5.3: Representation of the underlying system. 


explained in chaptci 3 we can approximate I{(^zi,Z 2 ) by polynomial transfer function 
using Equation error identification algorithm. 


5.5 Estimation of Depth and Displacement from the Ap- 
proximated Transfer Function. 


Once we arc able to approximate h{k, 1), it is easy to find the spread and displacement. 
As we see from (5.22) 


h{k,r) = 6{k,r) + h'{k,r)-k hd{k,l) 

or, h{k,l) - 5{k,l) - h'{k, 1) ★ hd{k, 1) (5.23) 


Let 

hi{k,l) = h'{k,l)i.hd{k,l) (5.24) 

Htiuation (5.24) has the same property as that of defocus operator hd{k,l]cr), except 
that it has the centroid shifted by (L*, Ly). In other words, equation (5.24) represents a 
dcfocus operator with center of mass at (Lj., Ly). Hence the center of mass of hi{k, 1) 
gives the displacements and its second moment gives the spread a. Hence, once we 
have h {k, 1) we can easily lind hi {k, 1) by subtracting 1 from h{0, 0). The displacements 
and the spread can then be calculated from its first and second moment respectively as 
defined in equations (5.5) and (5.6). 




Chapter 6 

Experimental Results 


I'hc mclliods discussed in the previous chapters have been implemented to test their 
cfticacy and accuracy. First, the results on motion estimation are discussed. We tested 
the motion-smear algorithm on simulated test pictures. Further, we restored the blurred 
image using the motion information as discussed in chapter 4 . Finally, we tested our 
algorithm for simultaneous estimation of motion and depth. 

6.1 Results on Simulated Smeared Image Frames 

Referring to equation ( 2 . 1 ), we have. 


1 

Jtx 

^ k =0 

where the interval is divided into T time steps. It means we can simulate 

g{x,y) by summing up unblurred images obtained at incrementally different camera 
translations corresponding to camera positions at instants . We adopted two methods 
to obtain two successive frames of motion smeared images. In the first method we take 
an image of size N x N of which only the region Ni x Ny {N i < N) is considered to 
avoid the boundary problems. From the iV x image, we created successive frames 
such that each frame is shifted by some pixels (say 1 ) w.r.t its previous frame in the 
x-direclion. Then the two required smeared frames 501 and 9x2 are obtained by adding 
the frames as discussed above. In a similar way motion-smear along both the x and y 



directions can be created. 

Figures 6.1, 6.3 and 6.5 show the results obtained during the testing of motion- 
smear algorithm on the test images simulated as discussed above. We selected a local 
region of size 41 x 41 assuming the maximum displacements to be 10 pixels in either 
directions. We then selected the order of the filter as discussed in chapter 3. Three 
cases have been considered. Fig. 6.1 shows the result of a cameraman image which is 
motion smeared by 5-pixels displacements along positive x-direction. In Fig. 6.3 the 
cameraman image is smeared by 6 -pixels displacements along both the direction. Fig. 
6.5 shows the motion smeared images with Lj = 6 and Lj, = 3. In each of the cases the 
PSF, optical flow and the plot of MSB vs. the value of M have been shown. As we see 
from tire mesh plot of the PSF, there are two impulses, one at the origin and the other 
one at {Lx, Ly) . Figs. 6.2 , 6.4 and 6.6 show the restored images using the regularized 
inverse filter with different values of 7 for each of the cases discussed above. 

Results for the noisy frames with SNR=25dB are shown in the fig. 6 . 7 . Figs. 6.7(a) 
and 6.7(b) are the two noisy motion-smeared frames with Lx = 5 and Ly = 0. We 
have estimated the PSF for both the cases (i.e, when both the input and output signals 
are denoised and when only input image is denoised). Figs. 6.7(c) and 6.7(d) show the 
PSFs for the two cases respectively. Fig. 6.7(e) shows the PSF estimated for the brick 
image with L* = 5 and Ly = 5 under noisy condition (SNR=25dB). The size of the 
local region is taken as 51 x 51. We have used Gaussian filter of size 3x3 and variance 
0.5 for denosing the signal. 'Fhe value of displacements calculated as first moment of 
the PSF arc shown in the figure itself. Figs. 6.7(g), 6.7(h) and 6.7(i) show the estimated 
optical flow for tlic three cases as mentioned above. The test image taken here is highly 
textured. The performance of the motion estimation algorithm on noisy textured image 
is better than that of noisy cameraman image. 

In the second method of data simulation, a shot of 1 second was taken from the 
digital camera connected to frame grabber board. In the shot, the person is holding a 
box which he moves slowly to his right during the shot with no body movement. This 
shot contains a sequence of 7 frames. The first three frames were added to create first 
frame qqi and the next three added to create gi^. Fig. 6.8 shows the results of motion 
estimation algorithm using the test images created by the shot of 1 second. The local 
area of size 51 x 51 was taken with Af = 5 for estimating the displacements. Two PSF 
has been shown in this result. One is for the area where there is no movement and the 
other is for the area containing the box. Lx and Ly calculated by analytical expression 
varies from Lx = - 1.2 to - 1-8 and Ly = 0.23 to 0.45 over the area containing the 



box and nearly zero at other parts. Fig. 6.8(d) shows the estimated optical flow over 
some points ot the image. The optical flow clearly shows some motion along negative 
x-direction near box area. 

Figs. 6.9 and 6.10 show the results for the algorithm developed to estimate motion 
and depth simultaneously. In this case two successive frames of smeared images goi and 
512 ^ are created. Motion smear is simulated by the first method as discussed earlier. For 
dcfocus wc have used the pill box funeflon with radius R = Z.The local region of size 
53 X 53 IS used for estimating the parameters assuming a maximum displacement of 10- 
pixcls in cither directions and a maximum blur circle of radius 3-pixels. Fig. 6.9 shows 
the result for the test images which contain both motion smear and defocus blur. The 
first frame is focused and motion-smeared by 4-pixcls displacements along positive x- 
dircction and the second frame is defocused and motion smeared. The radius used here 
is R = 3 pixels. The optical flow and the spread map are shown in figs. 6.9(e) and 
6.9(1). The mesh plot of PSF is shown in fig. 6.9(h) and the function hi(k, 1) obtained 
from tlte PSF is plotted in fig.6.9(i). As we clearly see from the plot that there is a circle 
shifted away from the origin. The amount of shift gives us the value of displacements 
and the radius of the circle is the blur circle radius. We calculated the displacements 
and Utc spread as Lx = 4.01 Ly = .012 and a = 2.167 analytically (as discussed in the 
chapter 5). 'Fhese values arc found to be close to their tme values. In fig. 6.10 the bark 
image is motion smeared by 6-pixcls along positive x and y direction. The amount of 
dcfocus used is same as in previous example. From the figure we note that the computed 
spread over tlie entire image is constant and very close to actual value ( the calculated 
cr = 2.165 ). 

The method discussed in [2] for estimating motion from motion-smear model uses 
exhaustive search method. It also uses a special sort of sensor arrangement to avoid in- 
stability problem. In our case we have considered a boxcar model. There is no problem 
in estimating stable PSF, as we are approximating the unknown transfer function by a 
polynomial transfer function. The algorithm proposed in [2] behaves good under noisy 
condition. In the present case, good denoising of the input signal is essential for better 
estimation of the PSF. Output noise doesn’t affect the estimation of transfer function 
coefficients as seen in chapter 3. We have tested our algorithm on a noisy and smeared 
textured image and obtained good results. All of the existing methods for image mo- 
tion estimation use displacement as a cue. Consequently, they all suffer from temporal 
aliasingl2]. There is no such problem of temporal aliasing in motion smear model. 

To the best of our knowledge, simultaneous estimation of motion and depth from 



smeared images has not been considered so far. 







Figure 6.1: (a) The original image of cameraman (b) Image poi, motion smeared by 
5 pixels displacement along positive x-direction (c) More smeared image S 02 (d) The 
estimated optical How over a portion of the image (e) The PSF of a local region size 
(41 X 4 1 ) with M = 6 (e) Plot of mean square equation error vs. M 
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Figure 6.2: Restored images with different value of gamma (a) 7 = .01 (b )7 = -001 
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(e) 

Meure 6.3: (a) Image r/oi, motion smeared by 6 pixels displacement along positive x 
and >' dircctu>n (b) The summed up signal (more smeared) (c) The PSF of a local region 
of siV.c 41 11 with .U " 7 (d) Plot of mean square equation error vs M (e) The 

estimated optical tlow over some portion of the image. 



I'igurc 6.4; Restored images with different value of regulanzmg parameter 7 (a) 7 .01 

(b) 7 .007 


38 



* ' 




39 



Figure 6.5; (a) Image tjau motion smeared with = 6 ,Lj, = 3 (b) Image 5112 (c) 
The sum signal .S 02 (d) The PSF of a local region of size 41 x 41 with M = 7 (e) The 
estimated optical flow over some points of the image, (f) Plot of MSB vs. the value of 
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Figure 6 . 6 ; Restored images using regularized inverse filter with (a) 7 .05 
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(i) 

Estimated displacements in (c) — 4.98 Ly = .025 (d) = 5.06 Ly = 0.016 (e) 

Lx = 4.879 Ly 4.96 

Figure 6.7; Results for the noisy case, (a) Noisy image 5oi„, (b) Summed up signal so 2 „ 
(c) PSF of the local region of size 51 x 51 with M = 6 and L^ = 5 Ly = 0 for the 
case when both the input and output signals are denoised. (d) PSF estimated when only 
input signal is denoised (e) Top view of the PSF of a local region with L^ = Ly = 5 
(0 Side view of the PSF (g) The estimated optical flow over some regions of the image 
for the case when both the signals are denoised (h) Optical flow diagram when only 
input is denoised (i) Optical flow diagram for the images, motion smeared by 5-pixels 
displacements along positive x and y direction. 
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Mgurc 6.9: (a) I'he original image of cameraman (b) Image motion smeared by 4 
pixels displacements along x-direction. (c) Image 512^, defocused and motion smeared 
(d) The sum sinnal S02. (e) The estimated optical flow over some points of the image 
(O The spread map (g) The PSF of a local region of size (53 x 53) with M = 8 (h) The 
function hi{kj){i) Plot of MSE vs. the value of M 
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Ingure 6 . HI: (:i) I iic original image of bark texture (b) Image 501 , motion-smeared by 
5 pixels i!i-pla ciiK-nt.s along both the direction (c) Image defocused as well as 
motion snicaicd id) Sum signal so 2 <i (e) The estimated optical flow over some points of 
the image <f) Spread map (g) The PSF h{k,l) of a local region of size (53 x 53) with 
” H ( h ) The t unction hi {k, 1) (i) Plot of mean square equation error vs. the value of 
M. 


47 


Chapter 7 


Conclusions and Scope for Future 
Work 


We ha\e propiKn! rnoUini estimatto'n !rom motion smeared images using the method 
ot ssstcni uicJiidieatutr.. i'iie inotuni estimated can also be used in image restoration 
b> using • ‘ ’ snseisc tiller, b'urthcr, the method can be reformulated to estimate 

motion and liepth • / - . \Vc see dial motion-smear and defocus blur, which 

are consideied a^ ‘ m the image, can be exploited to yield two very vital 

jMu.in,-; •' HU'tnm and depth. The simulation results show that the proposed method 
can estnn.ue depth, .md motum concetly by pro{KT selection of the window size and 
that ot the tudei id the . : pi'lsnoniial. It is also observed that estimation 

ot motion Hi till smem is less sensitive to temporal aliasing when compared to methods 
based i>n d ■ p'.n .• cues. 

Motion atid depth estimation from degradation is emerging area of research. Though 
a lot of utuk has been done in depth field using defocus concept, motion estimation 
from motion smear is a relatively new concept. The fusion of motion smear and defo- 
cus concept is really a very challenging work. New methods can be developed using 
multithunes i»r single trame. 
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